Machine interface

ABSTRACT

A machine interface allows a user to select a machine operation. A plurality of questions are stored for output to a user. A score indicating the likelihood that the user will select a machine operation is stored for each corresponding machine operation. A next question for output to the user is selected from the stored questions by determining, for each of a plurality of the questions, an average of the least number of questions required to be answered by the user to arrive at each machine operation weighted by the respective scores, and selecting the question having the lowest average number. The selected question is output and an answer is received from the user. In response to the input answer a machine operation is carried out and/or the stored scores for each of the plurality of machine operations is adjusted. At least one further selection of a next question is then carried out for output to the user using the adjusted stored scores. A set of data determined from at least one input answer may be stored identified by a bookmark so that it can be retrieved when the bookmark identifier is determined from an input answer.

[0001] The present invention generally relates to a machine interface toallow a user to select a machine operation from amongst a plurality ofpossible machine operations.

[0002] A great deal of effort has been expended in the prior art inorder to solve the problem of how to interface a machine to a user toenable a user to more readily control the functioning of a machine.

[0003] When there are a plurality of possible machine operations whichcan be carried out and a user cannot uniquely and immediately identifythe operation which the user requires to be carried out, it becomes aproblem as to how to interface the machine to the user to enable amachine to quickly and efficiently select a desired machine operation.For example, when accessing a database which contains retrievableinformation, a user may not know the exact identity of the data which isrequired, e.g. the file name of a picture or a document. The problem isthus how to interact with the user to extract the necessary informationto identify a required record.

[0004] This problem is also applicable to directing incoming calls in acall center. In such a system, a user wishes for a call to be routed toa particular destination but that destination is not known. Thus inorder to arrive at the target destination, a user must be prompted toinput information. This is typically achieved by asking questions of theuser in order to extract the necessary information.

[0005] A solution to this problem is disclosed in a paper entitled “AVector-Based Natural Language Call Routing” by J. Chu-Carroll and B.Carpenter (Computational Linguistics 1999). The solution described inthis paper is to receive a query from a user and to calculate andcompare a vector for the query with vectors for the nearest documents.If there is uncertainty about the nearest document vector, a new vectoris generated which would help to distinguish the nearest document. Thisvector is then used to generate a further question. The limitation ofthis system is that the confirming question only allows retrieval termsto be explicitly confirmed or rejected by the user.

[0006] In a machine interface which uses such a dialogue between amachine and a user, the user may often require to access the samemachine operation, e.g. the same database record, or have the calldirected to the same location. However, in order to return to the samepoint in a dialogue between the machine and the user, it is necessaryfor the user to repeat the dialogue. This can be a tedious operation forthe user.

[0007] One aspect of the present invention therefore overcomes theproblem of the prior art of interfacing a machine to a user wherein adialogue is entered into between the machine and the user in order toextract information from the user to identify a machine operation whichthe user wishes to implement. In accordance with the present invention aposition in the dialogue between the user and the machine can bebookmarked either automatically or manually by the user. The bookmarkcomprises a bookmark identifier which identifies a set of data which hasbeen arrived at a point in the dialogue between the user and themachine. The term bookmark refers to any machine readable label, flag,marker or identifier.

[0008] In one embodiment, the bookmarking can take place automaticallywhen a machine operation is executed after selection by the user. Thebookmark comprises a set of data derived from an initial input from theuser. The machine operation which was finally selected starting from theinitial user input is linked to a set of data derived from the initialuser input so that when the user once again generates a similar initialuser input the user can immediately cause the desired machine operationto be executed.

[0009] In one embodiment, scores for each of the machine operations isstored, where the scores indicate the likelihood that the user willselect a corresponding machine operation. Also scores for the or eachkeyword for each machine operation are stored, where the scores indicatethe likelihood that a user wishes to select a machine operation havingcaused a keyword to be input. When a user generates an input, keywordsare determined using the input and these are used to look upcorresponding scores which are then used to adjust the scores for themachine operations. The adjusted scores for the machine operations arethen used to identify a machine operation to be executed. The scores forthe keywords are also adjusted using the bookmark identifier.

[0010] In an alternative embodiment of the present invention, the usercan manually enter a bookmark at an appropriate position in the dialoguewith the machine. The manual bookmarking operation includes theinputting of a bookmark instruction and a bookmark identifier. Inresponse to the bookmark instruction, the current set of data determinedfrom the dialogue is stored identified by the bookmark identifier.

[0011] In an embodiment of the present invention, the set of data cancomprise keywords which can be used to look up scores for the keywordsfor each machine operation in order to modify stored scores for machineoperations. Alternatively, the set of data can comprise a set of scoresfor the machine operations as modified following the input of thekeywords. Either of these will result in a return to a position in thedialogue between the machine and the user marked by the bookmark.

[0012] In another embodiment of the present invention, the set of datacomprises pieces of input information. When the number of pieces ofinput information match the required number, an appropriate machineoperation can then be executed.

[0013] In the dialogue between the user and the machine, the questionsoutput to the user in order to extract information from the user can bedetermined based upon previous answers received from the user.Alternatively, the next question to be output to the user can simply beselected from a list of questions.

[0014] In the present invention, the machine operation can comprise anyoperation which can be carried out by a machine such as the retrieval ofdata, e.g. text, audio, video and images, or the execution of aninstruction such as the routing of incoming calls in a call center, theprinting of a document, or the transmission of a facsimile. Thus amachine operation can comprise any event which a user wishes to takeplace.

[0015] In an embodiment of the present invention, the plurality ofquestions for output to the user are stored questions which comprise alibrary of questions aimed at extracting a response from the user whichwill enable the system to uniquely identify the machine operation whicha user wishes to select. The stored questions can thus be tailored toprovide the most efficient selection of machine operations.

[0016] In an embodiment of the present invention, the stored questionsinclude expected answers. Any specified answer can have associated withit an identifier for a corresponding machine operation which is to becarried out in response to the input of the specified answer. Thus, eachmachine operation can have associated with it a “final” question whichwill allow the unique identification of the machine operation as theselected machine operation. The expected answers can have keywordsassociated therewith to allow for keyword matching with keywords for thetarget machine operations. Alternatively, instructions can be stored inassociation with the expected answers to extract keywords from thequestions.

[0017] In another embodiment of the present invention, a machineoperation is carried out in response to an answer when the score for themachine operation is significantly different from the scores for othermachine operations: thus indicating the unique identification of themachine operation. For example, the score for a machine operation may berequired to reach a threshold level greater than the other scores by athreshold amount.

[0018] One aspect of the present invention provides a machine interfacefor a machine which allows a user to select a machine operation. Themachine operation can comprise any operations which can be carried outby machine such as the retrieval of data e.g. text, audio, video andimages, or the execution of an instruction such as the routing ofincoming calls in a call center, the printing of a document, or thetransmission of a facsimile. Thus a machine operation can comprise anyevent which a user wishes to take place.

[0019] In the present invention a plurality of questions for output to auser are stored. This library of questions comprises a set of questionswhich aim to extract a response from the user which will enable thesystem to uniquely identify the machine operation which a user wishes toselect. The stored questions can thus be tailored to provide the mostefficient selection of machine operations. This flexibility allows for asystem administrator to modify the database of questions as desired.

[0020] A score is stored for each of a plurality of machine operations.The score indicates the likelihood that the user will select acorresponding machine operation.

[0021] A question for output to a user is selected from the storedquestions by determining, for each of a plurality of said questions, anaverage of the least number of questions required to be answered by auser to arrive at each machine operation. The average is a weightedaverage which is weighted by the respective scores for the machineoperations. The question having the lowest average number of questionsis then selected as the next question to be output to the user. Thusthis question selection process identifies a question which is likely tomost quickly result in the selection of a machine operation.

[0022] The selected question is then output to the user and an inputanswer is received in response. In response to a received input answer,a machine operation can be carried out. Alternatively or in addition,the scores for each of the stored plurality of machine operations areadjusted and a selection of a further question takes place using theadjusted scores. This process will repeat until the score for aparticular machine operation leads the question selection process to askthe user a question which enables a machine operation to be identifiedas the desired machine operation to be selected.

[0023] The present invention thus comprises a machine interface whichuses a plurality of questions to identify a desired machine operation.Questions to be asked of a user are adaptively selected based onprevious inputs by a user.

[0024] The advantage of the present invention is flexibility in thedesign since the number and type of questions can be tailored asrequired, which leads to more natural focussed and effective interactionwith the user.

[0025] In an embodiment of the present invention, the stored questionsinclude expected answers. Any specified answer can have associated withit an identifier for a corresponding machine operation which is to becarried out in response to the input of the specified answer. Thus, eachmachine operation will have associated with it a ‘final’ question whichwill allow the unique identification of the machine operation as theselected machine operation.

[0026] In an alternative embodiment, a machine operation is carried outin response to an answer when the score for the machine operation issignificantly different from the scores for other machine operations:thus indicating the unique identification of the machine operation. Forexample, the score for a machine operation may be required to reach athreshold level greater than the other scores by a threshold amount.

[0027] In one embodiment of the present invention, where expectedanswers to the questions are stored, question selection takes placeusing the determination of the least number of questions by predictingthe expected answers input by a user to select each of the machineoperations.

[0028] In an embodiment of the present invention, keywords are storedfor each of the machine operations and keywords are determined using theinput answer from the user. The system responds to the input answer bymatching the determined keywords to the stored keywords and adjustingthe scores for each of the plurality of machine operations in dependenceon the matching. Preferably, scores for the keywords for each of theplurality of machine operations are stored with the keywords. Scores forthe keywords determined from the input answers can then be determinedfor each of the plurality of machine operations by matching thedetermined keywords to the stored keywords. The scores of each of theplurality of machine operations are then adjusted using the determinedscores for the keywords.

[0029] Thus in the embodiment of the present invention, the questionsasked of the user are used to extract the necessary keywords from theuser in order to be able to perform a keyword search to identify adesired machine operation.

[0030] The keywords need not be input by the user. The keywords can bestored in association with the expected answers to at least some of thequestions. For example, a user may be asked the question ‘Do you wantmusic?’ and the expected answer would be ‘yes’ or ‘no’. The user has notentered keywords, but the keyword ‘music’ can be stored in associationwith the expected answer ‘yes’. This keyword can then be used forsearching.

[0031] In an alternative embodiment, instead of storing keywords inassociation with the expected answers, instructions can be stored forthe extraction of the keywords from the question.

[0032] The algorithm performed to determine the next question to ask auser is a recursive process in which sequences of questions to reacheach machine operation are processed in order to identify the shortestpath to each machine operation from each question. However, thisrecursive algorithm requires evaluation for all questions and for allmachine operations.

[0033] In a preferred embodiment, in order to reduce the processing,only machine operations having the highest scores are used in therecursive process.

[0034] In another embodiment of the present invention, in order toreduce processing, the path length i.e. the number of questions in asequence, is only allowed to reach a threshold length. Processing is notcarried on above a threshold question sequence length.

[0035] In a further embodiment of the present invention, the recursiveprocess is only performed for questions, the answers to which will causethe scores of a most likely machine operation to increase.

[0036] In yet another embodiment of the present invention, in order toreduce the number of questions in the recursive process, questions canbe preselected. Questions can be preselected on the basis of threecriteria:

[0037] (i) by taking the score of the machine instruction having thehighest score after asking the question and predicting a receivedanswer;

[0038] (ii) by assigning a high score to questions relating to the sametopic as a previous input answer; and

[0039] (iii) by assigning a high score to questions relating to the sametopic as any previous input answers.

[0040] In a preferred embodiment, questions are preselected on the basisof a weighted average of all three of these techniques.

[0041] In one embodiment of the present invention, the system is able toindicate to a user when it is operating with a high degree ofuncertainty e.g. when no machine operations have distinctive scores. Inorder to do this, scores for hierarchical classifications of the machineoperations are stored, where each hierarchical classification comprisesa topic to which the machine operations in the hierarchy below relateand each hierarchical classification has a score comprising the sum ofthe scores for the machine operations in the hierarchy below. When thescore for any of the hierarchical classifications at a predeterminedlevel of hierarchical classification is below a threshold, the systemcan indicate uncertainty to the user. This indication can help a user tomore carefully input a query (an answer to a question) which will morequickly result in the selection of a machine operation.

[0042] In order to allow a user to change the direction in which asearch for a desired machine operation is being carried out, in anembodiment of the present invention, scores for each of the machineoperations which is stored are decayed by a predetermined amount aftereach question has been answered. Thus, if a user changes the subject oftheir queries, the change in the score brought about by previous querieswill gradually decay thereby allowing a user's more recent answers topredominate in the search for the desired machine operation.

[0043] A feature of the present invention is the facility to enable thequestions to be added to and updated. Also, keywords used in anembodiment of the present invention can be added to and updated as wellas their scores.

[0044] The present invention is particularly suited to a dialogue systemin which a dialogue is entered into between a user and a machine inorder to achieve the implementation of the machine operation. Thepresent invention is particularly suited, although not limited to,implementation in a spoken dialogue system in which the questions aregenerated as a speech input and the answers are received as speech andprocessed by a speech recognizer.

[0045] The present invention can be implemented by dedicated hardware orby a suitably programmed processing apparatus, e.g. a programmed generalpurpose computer. The present invention thus encompasses computerprogram code for controlling a processor in a machine, e.g. in acomputer, to carry out the method. The present invention thusencompasses providing the computer code to the processing apparatus inany conventional form, such as: as a signal, e.g. an electrical signalcarried over a communications network such as the Internet, or on astorage medium such as a floppy disk, CD ROM, magnetic tape, or solidstate memory device. The computer program code can be provided on anysuitable carrier medium to the processing apparatus to be loaded in theprocessing apparatus to implement the method.

[0046] Embodiments of the present invention will now be described withreference to the accompanying drawings, in which:

[0047]FIG. 1 is a schematic diagram of a first embodiment of the presentinvention;

[0048]FIG. 2 is a schematic diagram of an implementation of the firstembodiment of the present invention on a general purpose computer;

[0049]FIG. 3 is a flow diagram illustrating the method of the firstembodiment of the present invention;

[0050]FIG. 4 is a schematic diagram of a second embodiment of thepresent invention;

[0051]FIG. 5 is a schematic diagram of an implementation of the secondembodiment of the present invention on a general purpose computer;

[0052]FIG. 6 is a flow diagram of the method of operation of the secondembodiment of the present invention;

[0053]FIG. 7A is a schematic illustration of a question data structure;

[0054]FIG. 7B is a schematic illustration of a “final” question datastructure; and

[0055]FIG. 8 is a schematic diagram of the hierarchical classificationsof records in accordance with an embodiment of the present invention;

[0056]FIG. 9 is a schematic diagram of an embodiment of the presentinvention;

[0057]FIG. 10 is a schematic diagram of an implementation of anembodiment of the present invention on a general purpose computer;

[0058]FIG. 11 is a flow diagram illustrating the method of accessing arecord of the database in accordance with an embodiment of the presentinvention;

[0059]FIG. 12 is a flow diagram of the process for selecting the nextquestion in accordance with an embodiment of the present invention; and

[0060]FIG. 13 is a flow diagram illustrating the path prediction step inthe flow diagram of FIG. 12 in more detail.

[0061] A first embodiment of the present invention will now be describedwith reference to FIGS. 1 to 3.

[0062] This first embodiment of the present invention comprises a manualbookmarking system wherein a user who enters into a dialogue with amachine is able to manually bookmark the position in the dialogue toenable the user to return to that position in the dialogue simply byinputting the bookmark.

[0063]FIG. 1 illustrates schematically this first embodiment of thepresent invention. The user is able to answer questions and enterbookmark information using the user input device 1. All input datacomprising answers to questions and bookmark information is received byan input translator 2. The input translator 2 translates the input dataeither into input data to be stored in the data structure which isstored in an input data store 4, or identifies uniquely a record whichcan then be retrieved from a record database 3 and output on an outputdevice 8. Also, the input translator 2 recognises an input bookmarkinstruction and stores the current data structure in the bookmark store5. Further, the input translator 2 recognises a request for retrieval ofbookmark data and accesses the stored data structure in the bookmarkstore 5. This retrieved data structure may enable the input translator 2to immediately retrieve a record from the record database 3 for outputby the output device 8. Alternatively, the retrieved data structure fromthe bookmark store 5 may simply be input into the input data store 4 incombination with any data structure already formed from input data.

[0064] Data within the input data store 4 is used by a question selector6 in order to select a question from a question database 7 to be outputby the output device 8. Thus in this embodiment, questions which areoutput to a user in order to prompt the user to input more data areselected based on previous input data by the user.

[0065]FIG. 2 illustrates the implementation of this embodiment of thepresent invention on a general purpose computer.

[0066] The computer includes an audio input device 20 such as amicrophone and suitable analogue-to-digital conversion means in order toinput spoken words into the computer. An audio output device 21, such asa loudspeaker, and suitable digital-to-analogue means is provided togenerate spoken words comprising questions or output audio data recordsto a user.

[0067] A question database 22 and a record database 23 are providedstored in conventional non-volatile memory means such as a hard diskdrive, CD ROM, floppy disk drive or solid state device. A working memory26 is provided to store data used during the implementation of thesystem. A program memory 27 is also provided to store the computerprogram code for the implementation of the system. The working memory 26and the program memory 27 can be provided on any conventional volatileor non-volatile memory means, e.g. hard disk drive, CD ROM, floppy diskdrive or solid state device. The computer program code can be providedto the program memory 27 using any conventional carrier medium. In FIG.2 a floppy disk drive 29 is illustrated. However, any other carriermedium such as a carrier signal, e.g. an electrical signal on theInternet, or any type of storage medium, e.g. CD ROM, tape device orsolid state device can be used.

[0068] A processor 25 is provided and comprises the conventional CPU ofa general purpose computer. The processor 25 implements variousfunctions by loading and running computer program code stored in theprogram memory 27. In the present embodiment, the processor 25implements a speech recognition engine 250 by loading and implementingspeech recognition engine code from the program memory 27. This enablesthe audio input received from the audio input device 20 to be convertedinto text. The processor 25 also implements an input translator 251 byloading and implementing input translator code from the program memory27. The input translator receives the output of the speech recognitionengine 250.

[0069] The processor 25 further implements a question selector 252 byloading and implementing question selector code from the program memory27. Also the processor 25 implements an audio output driver 253 byloading and implementing audio output driver code from the programmemory 27. The audio output driver 253 can cause the retrieval of audiodata as the selected record from the record database 25 by the audiooutput device 21. In an alternative arrangement, the audio output driver253 can include text-to-speech synthesiser if the records in the recorddatabase 23 comprise text. The text-to-speech synthesiser of the audiooutput driver 253 can then convert the text to speech data for output bythe audio output device 21.

[0070] The operation of the system will now be described with referenceto the flow diagram of FIG. 3.

[0071] In step S1 a question is selected for output to the user. In stepS2 the selected question is output to the user and in step S3 themachine waits for a user input. When a user input is received, in stepS4, the input translator 2 determines whether a bookmark instruction hasbeen received. If not, in step S5 the input translator 2 enters the dataand determines in step S6 whether the data structure is sufficient toidentify a record. If the data structure does identify a record, in stepS7 the record is retrieved from the record database 3 and output to theoutput device 8. In the implementation illustrated in FIG. 2, the recordretrieved comprises either an audio file or text which is converted toaudio for output by the audio output device 21.

[0072] If in step S6 the input translator 2 determines that the datastructure does not identify a record, in step S1 the question selector 6selects another question for output to the user.

[0073] If in step S4 it is determined by the input translator 2 that abookmark instruction is received, in step S8 it is determined whether abookmark retrieval instruction has been received. If the instruction isnot a retrieval instruction, in step S11 the input translator 2determines that a bookmark store instruction has been received and thusthe current data structure is stored in the bookmark store 5 indexed bythe bookmark identifier specified by the user. The process then returnsto step S6 to determine whether the data structure identifies a record.

[0074] If in step S8 it is determined that the bookmark instructionreceived comprises a retrieval instruction, the input translator 2 instep S9 retrieves bookmarked data from the bookmark store 5 using thebookmarked identifier associated with the retrieval instruction.

[0075] The retrieved data structure is then unified with any data in thecurrent data structure in step S10 by the input translator 2 by readingdata from the input data store 4 and the process returns to step S6 todetermine whether the data structure identifies a record.

[0076] In this embodiment of the present invention, a user is able tomanually request the bookmarking of a position in a dialogue with amachine such that a user can return to that position in the dialoguesimply by inputting a bookmark retrieval instruction with an associatedbookmark identifier.

[0077] In this embodiment of the present invention, the input data caneither be taken directly from the data input by the user, or from thequestion in association with the response by the user. Each expectedresponse can have input data associated with it or the identity of arecord which is to be accessed. Thus when an expected answer isreceived, this can either result in the generation of input data for theselection of the next question, or it can result in the identificationof a record for output.

[0078] Alternatively, the data structure can be built up by receivinginput pieces of information from the user input device 1. The inputtranslator 2 will thus continue to fill the data structure with piecesof information received until the data structure is complete. Thecomplete data structure will then identify a record which can beretrieved for output.

[0079] A second embodiment of the present invention will now bedescribed with reference to FIGS. 4 to 8.

[0080] In the second embodiment of the present invention, the systemautomatically generates bookmarks from an initial user input. When theuser initially inputs data, keywords are extracted from this to be usedas bookmarks for a record when this is finally identified and output.Thus the extracted keywords are used to increase scores for the keywordsor to add scores for the keywords so that when a user next inputs datawhich includes the keywords, the probability of quickly identifying therecord last identified by the keywords is improved.

[0081]FIG. 4 is a schematic illustration of the second embodiment of thepresent invention for accessing records in a database by receiving userinput queries and answers to questions generated by the system in orderto aid the identification of the desired record.

[0082] A user input device 101 receives user input. The user inputdevice 101 will provide text based on the input to an answer translator102 which interprets the answer by comparing the answer to expectedanswers to questions which are stored with the questions in a questiondata structure database 105. If the user input does not match theexpected answers for a question which was asked, the user input ispassed to a keyword extractor 104 to extract the keywords from the userinput. The keywords are then stored in a keyword list storage device103. If on the other hand the user input matches expected answers, thiscan result in the answer being translated to simply output a set ofkeywords associated with the expected answer to the keyword list storagedevice 103. If an answer which is matched to a user input indicates thatthe record should be rejected, the identity of the rejected record isstored in a rejected record storage device 106. If the user inputmatches an answer which has associated with it the identity of a recordwhich is to be selected, i.e. the user input is sufficient to identify arecord, the answer translator 102 will access the database for therecords 112 in order to cause the record to be retrieved and output toan output device 111. Each record of the database 112 has a score storedin an initial record scores database 113. The score for each recordindicates the likelihood that a user will wish to access the record. Theinitial scores can be used to identify popular records which are oftenaccessed by users.

[0083] A keyword scores database 108 is provided which stores a scorefor keywords for each record. Thus, for example, for a keyword “book”,scores for the keyword for records which have information on or relateto books will be high.

[0084] A score adjustment engine 107 is provided to read the keywordlist from the keyword list storage device 103 and to identify if anyrecords have been rejected by reading the rejected record storage device106. If any records have been rejected, their score is set to zeroindicating that the user does not wish to access these records.

[0085] The score adjustment engine 107 accesses the keyword scoresdatabase 108 using the keywords in the keyword list read from thekeyword storage device 103 in order to determine keyword scores forrecords. The score adjustment engine 107 also accesses current scoresfor records from a record scores storage device 109. Initially, thecurrent scores in the record scores storage device 109 can be set to theinitial record scores from the initial record scores database 113. Thescore adjustment engine then adjusts the current scores for each recordin dependence upon the scores determined for each keyword for eachrecord. The adjusted score is then stored as the current score for eachrecord in the record scores storage device 9.

[0086] The score adjustment engine 107 also receives initial keywordsidentified from an initial input from a user by the answer translator102. The initial keywords are used as bookmarks in order to adjust thekeyword scores for the records in the keyword scores database 108.

[0087] When a record has not been identified as a desired record as aresult of a user input, the system requires more information to enableit to identify a desired record. This information is obtained by askingthe user a next question retrieved from the question data structuredatabase 105. The next question to be selected to be retrieved from thequestion data structure database 105 is determined by a questionselector 110 which selects the question on the basis of the currentscores for each record stored in the record scores storage device 109.Once a question has been selected by the question selector 110, it isretrieved from the question data structure database 105 and output tothe output device 110.

[0088] Thus the embodiment of the present invention will continue to askquestions selected by the question selector 110 of the user in order toextract more keywords which will help to identify a desired record byadjusting the scores appropriately for the records.

[0089]FIG. 5 is a schematic diagram of an implementation of the secondembodiment of the present invention in a general purpose computer whichinterfaces to a user using speech.

[0090] The computer includes an audio input device 120 such as amicrophone and suitable analogue-to-digital conversion means in order toinput spoken words into the computer. An audio output device 121 such asa loudspeaker and suitable digital-to-analogue means is provided togenerate spoken words comprising questions or output audio data recordsto a user.

[0091] A question database 122, a record database 123 and a keyworddatabase 124 are provided stored in conventional non-volatile memorymeans such as a hard disk drive, CD ROM, floppy disk drive or solidstate device. A working memory 126 is provided to store data used duringthe implementation of the system. A program memory 127 is also providedto store the computer program code for the implementation of the system.The working memory 126 and the program memory 127 can be provided on anyconventional volatile or non-volatile memory means, e.g. hard diskdrive, CD ROM, floppy disk drive, or solid state device. The computerprogram code can be provided to the program memory 127 using anyconventional carrier medium. In FIG. 5 a floppy disk 129 is illustrated.However, any other carrier medium such as a carrier signal, e.g. anelectrical signal carried over the Internet, or any type of storagemedium, e.g. CD ROM, tape device, or solid state device can be used.

[0092] A processor 125 is provided and comprises the conventional CPU ofa general purpose computer. The processor 125 implements variousfunctions by loading and running computer program code stored in theprogram memory 127. In the present embodiment, the processor 125implements a speech recognition engine 1250 by loading and implementingspeech recognition engine code from the program memory 127. This enablesthe audio input received from the audio input device 120 to be convertedinto text. The processor 125 also implements an answer translator 1251by loading and implementing answer translator code from the programmemory 127. The answer translator 1251 receives the output of the speechrecognition engine 1250.

[0093] The processor 125 further implements a keyword extractor 1252 byloading and implementing keyword extractor code from the program memory127. Also the processor 125 implements a score adjustment engine 1253 byloading and implementing score adjustment engine code from the programmemory 127. Further, the processor 125 implements a question selector1254 by loading and implementing question selector code from the programmemory 127. Also, the processor 125 implements an audio output driver1256 by loading and implementing audio output driver code from theprogram memory 127. The audio output driver 1256 can cause the retrievalof audio data as the selected record from the record database 125 foroutput by the audio output device 121. In an alternative arrangement,the audio output driver 1256 can include a text-to-speech synthesiser ifthe records for the record database 123 comprise text. Thetext-to-speech synthesiser of the output audio driver 1256 can thenconvert the text-to-speech data for output by the audio output device121.

[0094] The operation of the system will now be described.

[0095] The records of the database of this embodiment comprise audiofiles in the “wave” file format. Each record is identified by a recordnumber to allow for ease of access.

[0096] The question data is formed into question data structures asillustrated in FIGS. 7A and 7B. Each question is identified by aquestion number. Associated with the question is a question prompt as anaudio file in the “wave” format, e.g. in QUESTION10.WAV. Associated witheach question are expected answers. In the embodiment illustrated inFIG. 7A, the expected answers are “yes” or “no”. The question output inthis example could be an audio question “Do you want pop music?”. If theuser answers “yes”, associated with the expected answer “yes” are thekeywords “pop music” and “rock music”. If the user answers “no”,associated with the expected answer “no” is an instruction to rejectthree records as not being records which will be desired by the user,i.e. records 18, 22 and 36. This list of rejected records is stored inthe rejected record list. The question data structure also includes anindication of the topic of the question which in this case generallycomprises the topic “music”.

[0097] The question data structure illustrated in FIG. 7A comprises aquestion data structure which does not result in the selection of arecord as a result of an answer. Instead, the answer will result in therejection of some records and the input of keywords which can be used toadjust the scores for records which will then be used to select the nextquestion to ask the user.

[0098]FIG. 7B illustrates another question data structure which istermed the “final” question data structure for a record. The questiondata structure is the same as that of FIG. 7A except in the examplegiven, the question to be output to the user is of course a differentaudio file related to question number 15. Also the expected answersresult in different operations. For example, the question could be “Doyou want pop artist 1?”, where record number 20 contains a piece ofmusic by pop artist 1. If the answer to this question is “yes”, in thequestion data structure there is an instruction to set the selectedrecord identifier for record number 20.

[0099] If on the other hand the answer is “no”, the rejected recordidentifier is set to record number 20.

[0100] The other difference between the question data structure of FIG.7A and the question data structure of FIG. 7B is that the topic is morenarrowly defined as “pop music”.

[0101]FIG. 6 is a flow diagram illustrating the operation of thisembodiment of the present invention.

[0102] In step S20 an initial question is output to the user. Thisquestion can simply be an initial prompt, e.g. “What would you like?”and the scores for the records are set to the initial record scores. Instep S21 the system awaits the user input and when this is received, instep S22, the answer translator determines whether the input matches anexpected answer. If it does not, in step S24 keywords are extracted fromthe input. In step S24 it is then determined whether the input is aninitial input. If so in step S25 keywords for the initial input arestored to act as bookmarks. Then in step S26 the keywords are added tothe keyword list and in step S27 the keywords are used to search in thekeyword database the scores for the words for each record. These scoresare then used to determine a revised score for each record.

[0103] In step S28 a next question to ask the user is selected using therevised scores for each record. The selected question is then output instep S29 to the user and the process returns to step S21 to await theuser input.

[0104] In this embodiment the initial scores for the records are set asan initial probability p(x). The scores for keywords stored comprises aprobability of a word given a record p(w|x). The probability is thusupdated by multiplying the current probability p(x) by the wordprobability p(w|x).

[0105] In order to take into account the possibility that a user changesthe target record during the question and answer session, the currentprobability for records is allowed to decay back towards the initialprobability. For example, the new probability can be calculated from:

p(x)=(0.2×p _(i)(x)+0.8×p(x))×p(w|x)

[0106] where p_(i)(x) is the initial probability.

[0107] It can be seen that with a decay set by the numbers 0.2 and 0.8,the current probability can be made to decay towards the initialprobability if the word probability does not modify the currentprobability.

[0108] If in step S22 the user's input does match an expected answer,the answer translator, in step S30, translates the input to keywords ifthe answer has keywords associated with it. Alternatively, if there isan instruction associated with an expected answer to set the selectedrecord identifier to a record number or to set the rejected recordidentified to a record number, this is done. Then in step S31 it isdetermined whether the selected record identifier identifies a record.If not, in step S32 it is determined whether the rejected recordidentifier identifies a record. If not, this means that the answertranslator has determined keywords associated with the answer and theseare added to the keyword list in step S26 to be used in step S27 torevise the score for each record. The revised score can then be used instep S28 to select the next question to ask a user for output in stepS29.

[0109] If in step S32 it is determined that there is a record identifiedby the rejected record identifier, in step S33 the score for the recordis fixed to zero and the process to select the next question to ask auser in step S28 is carried out with the score for the record fixed tozero. If step S33 has been carried out a number of times, there can be anumber of rejected records listed for which the scores are fixed tozero. These are listed in the rejected record list to ensure that theirscores remain fixed at zero in the current scores used by the questionselector to determine the next question to ask a user. If in step S31 itis determined that a record is identified, in step S34 any initialkeywords stored in step S25 are stored as keywords for the identifiedrecord. Then in step S35 the record is retrieved.

[0110] Thus in this embodiment of the present invention the keywordsdetermined from an initial input from the user are used as bookmarks byadding the keywords to keywords for a record which was finallyidentified as the target record starting from the initial query from theuser.

[0111] In this embodiment the keywords in the keyword database haveprobabilities associated with them. Thus in step S34, rather than simplyadding keywords to the list, the probabilities for the keywords listedin the keyword database is modified. For example, where the keyword doesnot appear in the keyword database, the keyword is added to the databasewith a high probability. If the keyword is already in the database, itsprobability is increased. Since probability is used in this embodimentfor the words, and since all the probabilities must add up to 1.0, wherekeywords identified in step S25 have their probabilities increased,probabilities for other words must decrease. A set of probabilities forkeywords can be stored as a bookmark to be used for the calculation ofprobabilities for records. The probabilities for records are themselvesused to identify the record being sought by the user.

[0112] In this embodiment, any method of selecting a question can beused, e.g. simply by referring to the topics of the questions.Preferably, however, this embodiment of the present invention implementsthe technique for selecting the question disclosed in a copending UKapplication by the same assignee filed on Mar. 29, 2000, the contents ofwhich are hereby incorporated by reference. The particular technique forselecting the next question is not an essential feature of the presentinvention. For example, the next question could simply be selected froma list of questions.

[0113] In the foregoing embodiments, probabilities have been describedas usable with keywords for records. It is, however, also possible tobookmark not just the final records to be selected, but also anyposition in a dialogue. In one embodiment this can be achieved by ahierarchical set of questions so that the probability for the recordscan be formed into a hierarchical tree as illustrated in FIG. 8.

[0114] In FIG. 8 the numbers indicate probabilities for records, andclassifications of records. The keywords “music” and “cars” can be usedas a third hierarchical level and the keywords “classical”, “pop”,“British” and “foreign” can be used as keywords at a second hierarchicallevel of classification. Thus, for instance, after having reached theclassification “pop”, which identifies two possible records “artist 1”and “artist 2”, a user may wish to bookmark this point of the dialogue,e.g. using the spoken word “jack”. Thus the bookmark identifier “jack”will identify keywords which, in this example, can be the keywords“music” and “pop”. These keywords will have probabilities associatedwith them which will result in the probabilities for the final records“artist 1” and “artist 2” being adjusted to 0.2 and 0.3 which is higherthan the other records.

[0115] This enables a user to return to a point in the dialogue whichcould then result in a question being asked “Do you want artist 1 orartist 2?”. Thus, this position in the dialogue is bookmarked withouthaving to actually bookmark a record. This avoids the user having toreach a midpoint in a dialogue by repeating previous dialogue steps,e.g. answering the question “Do you want music or cars?” and “Do youwant classical music or pop music?”.

[0116]FIG. 9 is a schematic illustration of an embodiment of the presentinvention for accessing records in a database by receiving user inputqueries and answers to questions generated by the system in order to aidthe identification of the desired record.

[0117] The user input device 1 receives user input. The input devicewill provide text based on the input to an answer translator 30 whichinterprets the answer by comparing the answer to expected answers toquestions which are stored with the questions in a question datastructure database 33. If the user input does not match the expectedanswers for a question which was asked, the user input is passed to akeyword extractor 32 to extract the keywords from the user input. Thekeywords are then stored in a keyword list storage device 31. If on theother hand the user input matches expected answers, this can result inthe answer being translated to simply output a set of keywordsassociated with the expected answer to the keyword list storage device31. If an answer which is matched to a user input indicates that therecord should be rejected, the identity of the rejected record is storedin a rejected record storage device 34. If the user input matches ananswer which has associated with it the identity of a record which is tobe selected, i.e. the user input is sufficient to identify a record, theanswer translator 30 will access a database for the records 39 in orderto cause the record to be retrieved and output to an output device 8.

[0118] Each record of the database 39 has a score stored in an initialrecord scores database 40. The score for each record indicates thelikelihood that a user will wish to access the record. The initialscores can be used to identify popular records which are often accessedby users.

[0119] A keyword scores database 36 is provided which stores a score forkeywords for each record. Thus, for example, for a keyword ‘book’,scores for the keyword for records which have information or relate tobooks will be high.

[0120] A score adjustment engine 35 is provided to read the keyword listfrom the keyword list storage device 31 and to identify if any recordshave been rejected by reading the rejected records storage device 34. Ifany records have been rejected, their score is set to zero indicatingthat the user does not wish to access these records.

[0121] The score adjustment engine 35 accesses the keyword scoresdatabase 36 using the keywords in the keyword list read from the keywordstorage device 31 in order to determine keyword scores for records. Thescore adjustment engine 35 also accesses current scores for records froma record scores storage device 37. Initially, the current scores in therecord scores storage device 37 can be set to the initial record scoresfrom the initial record scores database 40. The score adjustment engine35 then adjusts the current scores for each record in dependence uponthe scores determined for each keyword for each record. The adjustedscore is then stored as the current score for each record in the recordscores storage device 37.

[0122] When a record has not been identified as a desired record as aresult of a user input, the system requires more information to enableit to identify a desired record.

[0123] This information is obtained by asking the user a next questionretrieved from the question data structure database 33. The nextquestion to be selected to be retrieved from the question data structuredatabase 33 is determined by a question selector 38 which selects thequestion on the basis of the current scores for each record stored inthe record scores storage device 37. Once a question has been selectedby the question selector 38, it is retrieved from the question datastructure database 33 and output to the output device 8.

[0124] A re-configuration interface 41 is provided and allows for theadjustment of the initial scores for records in the initial recordscores database 40. The scores for records can be adjusted to take intoaccount changes in user behaviour, e.g. the popularity of particularrecords, or to add initial scores for records which have been added tothe database of records 39.

[0125] The re-configuration interface 41 is also provided to allow anadministrator or manager of the system to reconfigure any of the data inthe databases 33, 36, 39, 40. This provides a system with a great dealof flexibility since it allows the records which can be selected by auser to be updated. It also allows the questions to be updated asnecessary either to improve the abilities of the system to identify arecord, or to add new questions when new records are added to thedatabase records 39. Also, the re-configuration interface 41 allows thekeyword scores database 36 to be updated to take into account changes inuser behaviour and changes in the records in the database of records 39.

[0126] Thus this embodiment of the present invention will continue toask questions selected by the question selector 38 of the user in orderto extract more keywords which will help to identify a desired record byadjusting the scores appropriately for the records.

[0127]FIG. 10 is a schematic diagram of an implementation of the systemof FIG. 9 in a general purpose computer which interfaces to a user usingspeech.

[0128] The computer includes an audio input device 320 such as amicrophone and suitable analogue to digital conversion means in order toinput spoken words into the computer. An audio output device 321 such asa loudspeaker and suitable digital to analogue means is provided togenerate spoken words comprising questions or output audio data recordsto a user.

[0129] A question database 322, a record database 323, and a keyworddatabase 324 are provided stored in conventional non-volatile memorymeans such as a hard disc drive, CD ROM, floppy disc drive or solidstate device. A working memory 326 is provided to store data used duringthe implementation of the system. A program memory 327 is also providedto store the computer program code for the implementation of the system.The working memory 326 and the program memory 327 can be provided on anyconventional volatile or non-volatile memory means e.g. hard disc drive,CD ROM, floppy disc drive or solid state device. The computer programcode can be provided to the program memory 327 using any conventionalcarrier medium. In FIG. 10 a floppy disc drive 329 is illustrated.However, any other carrier medium such a carrier signal e.g. anelectrical signal on the Internet, or any type of storage medium e.g. CDROM, tape device, or solid state device can be used.

[0130] A processor 325 is provided and comprises the conventional CPU ofa general purpose computer. The processor 325 implements variousfunctions by loading and running computer program code stored in theprogram memory 327. In the present embodiment, the processor 325implements a speech recognition engine 3250 by loading and implementingspeech recognition engine code from the program memory 327. This enablesthe audio input received from the audio input device 320 to be convertedinto text. The processor 325 also implements an answer translator 3251by loading and implementing answer translator code from the programmemory 327. The answer translator 3251 receives the output of the speechrecognition engine 3250.

[0131] The processor 325 further implements a keyword extractor 3252 byloading and implementing keyword extractor code from the program memory327. Also the processor 325 implements a score adjustment engine 3253 byloading and implementing score adjustment engine code from the programmemory 327. Further, the processor 325 implements a question selector3254 by loading and implementing question selector code from the programmemory 327. Also, the processor 325 implements an audio output driver3256 by loading and implementing audio output driver code from theprogram memory 327. The audio output driver 3256 can cause the retrievalof audio data as the selected record from the record database 325 foroutput by the audio output device 321. In an alternative arrangement,the audio output driver 3256 can include a text to speech synthesiser ifthe records in the record database 323 comprise text. The text to speechsynthesiser of the audio output driver 3256 can then convert the text tospeech data for output by the audio output device 321.

[0132] The processor 325 also implements a re-configuration interface3257 by loading and implementing configuration interface code from theprogram memory 327. The re-confirmation interface can be used forre-configurating the data in any of the databases 322, 323, 324.

[0133] The operation of the system will now be described.

[0134] The records in the database in this embodiment also compriseaudio files in the “wave” file format as described above with referenceto FIGS. 7A and 7B.

[0135]FIG. 11 is a flow diagram illustrating the operation of the systemaccording to this embodiment of the present invention.

[0136] In step S100 an initial question is output to the user. Thisquestion can simply be an initial prompt, e.g. “What would you like?”and the scores for the records are set to the initial record scores. Instep S101 the system awaits the user input and when this is received, instep S102, the answer translator determines whether the input matches anexpected answer. If it does not, in step S103 keywords are extractedfrom the input and in step S104 the keywords are added to the keywordlist. The keywords are then used in step S105 to search in the keyworddatabase for scores for the words for each record. These scores are thenused to determine a revised score for each record.

[0137] In step S106 a next question to ask a user is selected using therevised scores for each record. The selected question is then output instep S107 to the user and the process returns to step S101 to await auser input.

[0138] In this embodiment the initial scores for the records are set asan initial probability p(x). The scores for keywords stored comprises aprobability of a word given a record p(w|x). The probability is thusupdated by multiplying the current probability p(x) by the wordprobability p(w|x).

[0139] In order to take into account the possibility that a user changesthe target record during the question and answer session, the currentprobability for records is allowed to decay back towards the initialprobability. For example, the new probability can be calculated from:

p(x)=(0.2×p _(i)(x)+0.8×p(x))×p(w|x)

[0140] where p_(i)(x) is the initial probability.

[0141] It can be seen that with a decay set by the numbers 0.2 and 0.8,the current probability can be made to decay towards the initialprobability if the word probability does not modify the currentprobability.

[0142] If in step S102 the user's input does match an expected answer,the answer translator, in step S108, translates the input to keywords ifthe answer has keywords associated with it. Alternatively, if there isan instruction associated with an expected answer to set the selectedrecord identifier to a record number or to set the rejected recordidentifier to a record number, this is done. In step S109 it is thendetermined whether the selected record identifier identifies a record.If so, this means that a record has been selected and in step S110 therecord is output. If in step S109 no record is identified by theselected record identifier, in step S111 it is determined whether therejected record identifier identifies a record. If not, this means thatthe answer translator has determined keywords associated with the answerand these are added to the keyword list in step S104 to be used in stepS105 to revise the score for each record. The revised score can then beused in step S106 to select the next question to ask a user for outputin step S107.

[0143] If in step S111 it is determined that there is a recordidentified by the rejected record identifier, in step S112 the score forthe record is fixed to zero and the process to select a next question toask a user in step S106 is carried out with the score for the recordfixed to zero. If step S112 has been carried out a number of times,there can be a number of rejected records listed for which the scoresare fixed to zero. These are listed in the rejected record list toensure that their scores remain fixed at zero in the current scores usedby the question selector to determine the next question to ask a user.

[0144] The method by which the next question is determined using thescores for each record will now be described with reference to the flowdiagrams of FIGS. 12 and 13.

[0145] The algorithm to identify the next question is based on arecursive process wherein for each question and for each record anestimate is made as to the least number of questions needed to arrive atthe record after answering the question.

[0146] The algorithm for selecting the next question to ask the user isbased on an optimal or best answer assumption. Each question hasassociated with it expected answers. The assumption is that if aquestion is asked, the user will give the best answer to reach a targetrecord.

[0147] Because the expected answers to the questions are known, it ispossible to generate a sequence of questions and predicted answers inorder to reach a target record. Using these predicted answers it ispossible to select a good path, i.e. a path having the least number ofquestions for every record. Thus the algorithm operates by looking ateach question and predicting a response. The predicted response is thenused to calculate predicted scores for the records and the predictedscores for the records are then used to select a next question. Thisprocess repeats to find paths using a sequence of questions andpredicted answers to reach each record. The shortest path length reachrecord is selected and an average of the shortest path lengths is takenwherein the average is weighted by the current probability for eachrespective record.

[0148] This process will be described in more detail with reference toFIGS. 12 and 13.

[0149] In step S201 the question index Q is set to 1 to start the firstquestion, and in step S202 the record index x is set to 1 to start thefirst question. In step S203, assuming a user wishes to retrieve recordx, the path is predicted after question Q in order to retrieve questionx and the path length is stored as DL(x). Then in step S204 the recordindex is incremented and in step S205 it is determined whether there aremore records to process and if so the process returns to step S203 topredict path lengths DL(x) for these records. Once all the records havebeen processed in step S206, the path length for the question is takenas the weighted average of the shortest path lengths to each record. Theweighting used is the probability for each respective record. Theequation for the calculation of the path length for each question isgiven by:

Σp(x)DL(x).

[0150] In step S207 the question index Q is incremented and in step S208it is determined whether all questions have been evaluated. If not theprocess returns to step S202 to continue the evaluation for eachquestion and for each record. If all questions have been evaluated todetermine the weighted average of the shortest path lengths, in stepS209 the question is selected which has the shortest path length.

[0151] Thus for each question the path length is determined as anaverage path length taking into account the likelihood that the userwill take a particular path to a particular record (because it'sprobability is higher). Thus the process uses a statistical process toselect the most suitable next question to arrive most quickly at arecord.

[0152] The process of predicting the path to a record in step S203 ofFIG. 12 will now be described in more detail with reference to FIG. 13.

[0153] In step S300 an answer is chosen which a user would give toarrive at the record x. The predicted answer is then translated in stepS301 to look up the keywords or action relating to the answer. If instep S302 the action selects a record, in step S303 a path length equalto 1 is returned. If a record is not selected, in step S304 the keywordsare used to update the predicted probability for the record p(x). Theprocess then has to select the next question in the path and this isdone by initially setting the question index to 1 and predicting thepath length again using the same process as step S203. In step S307 thepredicted path length DL(x) is stored and in step S308 the questionindex is incremented. In step S309 it is then determined whether all ofthe questions have been processed and if not the process returns to stepS306. If all of the questions have been processed, in step S310, theshortest path length is selected and in step S311 the shortest pathlength +1 is returned.

[0154] Thus FIGS. 12 and 13 comprise a recursive process used toidentify path lengths for each next question to each record, where thepath length comprises a sequence of questions of shortest length.

[0155] In the algorithms described with reference to FIGS. 12 and 13,the processing can be quite demanding where there are a large number ofquestions and/or a large number of records. The following embodimentsdescribe optimisation techniques in order to reduce the processingrequirements of the algorithm.

[0156] In a first optimisation embodiment, instead of including allrecords in the algorithm, at each loop in the algorithm only recordswhich are likely to be selected by the user are processed. These recordsare identified by their probabilities. Thus only records which have aprobability above a threshold are included in the list of records ateach loop of the procedure. This avoids having to process unlikelyrecords.

[0157] A second optimisation embodiment comprises limiting the recursiondepth. The number of questions included in the path length can be set toa threshold, e.g. 10. Once the algorithm recursively calculates thelength DL(x) as reaching 10, it can stop and simply return a maximumvalue, i.e. 10. This assumes that there will be some other questionwhich will provide a lower expected path length DL(x) which will beselected.

[0158] A third optimisation embodiment comprises only selectingquestions which are likely to have a positive effect. Since thequestions have expected answers and actions or keywords associated withthe expected answers, the predicted effect on the record having thehighest probability can be determined. If the expected answer to aquestion generates keywords which can increase the probability for therecord with the highest probability, this indicates that it could be auseful question to ask. In this way the number of questions which haveto be processed can be reduced by ignoring questions which cannotincrease the probability of the highest probability record.

[0159] A fourth optimisation embodiment involves the pre-selection ofquestions. Thus rather than calculating the path DL(x) for everyquestion, it is only calculated for a small number of questions. Thequestions can be pre-selected by giving the questions a score and onlyselecting the questions which have a score above a threshold orselecting a group of questions which have the highest scores.

[0160] There are many ways in which scores can be attributed toquestions. One way is to assign a question the probability for therecord having the highest probability after answering the question.Another method is to assign a high score to questions which relate tothe same topic as a previous dialogue, i.e. question and an answer.Another method is to assign a high score to questions that relate to thesame topic as all previous dialogues, i.e. questions and answers.

[0161] Questions that relate to the same topic can be determined bycomparing the topic data entry for the question (see FIGS. 7A and 7B)with the topic data entry for one or more previous user inputs.

[0162] Thus by assigning scores to questions it is possible to selectquestions with a score over a threshold, or a fixed number of questionswith the highest score and calculations need only be carried out forthese.

[0163] Any one of these processes for calculating probabilities orscores for questions can be used alone or in combination. For example, aweighted average of the three determined scores can be assigned to eachquestion.

[0164] A further feature of an embodiment of the present invention willnow be described with reference to FIG. 8.

[0165] Imagine a user says “I want Salsa” but the speech recognition forthis misunderstands it as “I one these are”. These can easily happenbecause of noise, deformations of the voice over a telephone line,unclear pronunciation or accent and many other reasons. In this case thesystem will not have good terms for deciding what the user wants.

[0166] Also, the user could change interest during the dialogue with thesystem e.g. a user first asks for Salsa and later for jazz. Further, auser input such as an initial input could lack useful keywords e.g. “Iwould like to buy a record with some dance music”. The keywords “dancemusic” may not be particularly distinctive if there are thousands ofrecords relating to dance music.

[0167] It can thus be useful to indicate to the user that the system isuncertain, or in other words that the supplied terms are not effectivein differentiating between records. The system can indicate uncertaintyby feedback to the user such as by saying “I am not very sure. Do youwant classical music?”.

[0168] Thus this further feedback to a user can help a user to try tothink of a more useful input to identify the record being sought.

[0169] In this embodiment records are organised in a hierarchicalstructure by calculating the similarity between records e.g. classifyingthe records. This can result in a tree as illustrated in FIG. 8 whereinthe records comprise the leaves of the tree. Every node in the tree isassigned a value calculated as the sum of the probabilities or scores ofthe leaves of which it is a parent. In FIG. 8 probability values areused and hence at any level in the hierarchy the probability adds up to1.0.

[0170] Thus in the storage of the current record scores, scores fornodes of the hierarchical tree can also be stored to facilitate thefeature of indicating uncertainty to a user. The probability values foreach node can be adjusted in accordance with the adjustments made to theprobability values for the records in the hierarchy below the nodes.

[0171] This hierarchical structure of probabilities can be used forindicating uncertainty to a user by setting a threshold probabilityvalue, e.g. 0.6 shown in FIG. 8. Using this threshold probability value,nodes in the hierarchical structure which have probabilities above andbelow the threshold can be identified. If no nodes can be identifiedhaving a probability above the threshold value at a suitable level inthe hierarchy, this indicates that no distinctive records have beenidentified and uncertainty can be indicated to a user.

[0172] The embodiment described above with reference to FIGS. 1 to 8 maybe modified by adding the reconfiguration interface shown in FIG. 9 andthe processor configured to enable the method illustrated by FIGS. 11 to13 to be carried out in addition to the method illustrated by FIG. 6 sothat the bookmark facility described with reference to FIGS. 1 to 8 maybe implemented with the question determining procedure described withreference to FIGS. 9 to 13, 7A, 7B and 8.

[0173] Although the present invention has been described hereinabovewith reference to specific embodiments, modifications will be apparentto a skilled person in the art which lie within the spirit and scope ofthe present invention.

[0174] Although in the embodiments probabilities used for words andrecords, any form of score can be used.

[0175] As described above, a “final” question is used to cause theselection of a record for output. However, the selection of a record foroutput can alternatively take place by selecting a record which has ascore which is significantly high, e.g. as a score above a thresholdwhich is greater than other scores by a threshold amount.

[0176] As described above, the bookmark comprises a spoken bookmark (amanual bookmark). The bookmark can, however, be manually input in anyway, e.g. using a keyboard.

[0177] As described with reference to FIG. 11, expected answers havekeywords, rejected records or selected records associated with them. Thepresent invention also encompasses a combination e.g. keywords andrecords to be rejected in response to the associated answer.

[0178] The present invention is applicable to any means by whichquestions and answers can be conveyed to and from a user to the system.The user interface can comprise speech or text for example.

[0179] The present invention is applicable to the selection of any typeof machine operation from a number of possible machine operations. Forexample, the present invention is applicable to the selection of datarecords for retrieval, e.g. the retrieval of images, text, audio andvideo. Alternatively, the machine operation can simply comprise themarking or identification of a selected record. Further, the machineoperation can be the selection and execution of a spoken dialogue modulesuch as a VOXML file. Also, the present invention is applicable to callcenter technology wherein the selected machine operation is the routingof a telephone call or the selection of a service. Further, the presentinvention is applicable to telephone banking wherein the selection ofbanking services and banking information can be achieved.

[0180] Embodiments of the present invention can be intended to providebookmarks for individual users and can thus provide individual userprofiles.

[0181] The present invention can be implemented by dedicated hardwareconfigured to perform the functions of the system.

[0182] More preferably, the present invention is implemented in aprocessing system by computer program code. Such a processing system canbe provided in any form of apparatus, such as in a photocopying machine,facsimile machine, mobile telephone, or a general purpose computer.

[0183] The present invention thus encompasses program code forcontrolling a processor to implement a method. The program code can beloaded into the processing system from any conventional carrier mediumsuch as a transient carrier medium (e.g. an electrical signal carryingthe program code) or a storage medium, such as a floppy disk drive, CDROM, magnetic tape device or solid state device.

1. A machine having a machine interface to allow a user to select amachine operation, the machine interface comprising: outputting meansfor outputting questions to the user; inputting means for receivinginput answers to the questions; and processing means for determining aset of data from at least one said input answer and for using the set ofdata to execute a said machine operation and for determining a questionfor output by said outputting means, said processing means being adaptedto store a set of data determined from at least one said input answeridentified as a bookmark and to retrieve said set of data when thebookmark identifier is determined from a said input answer.
 2. A machineaccording to claim 1, including keyword determining means fordetermining keywords using the input answer, wherein said set of datacomprises at least one keyword and said bookmark identifier comprisessaid at least one keyword.
 3. A machine according to claim 2, includingscore storage means for storing a score for each of a plurality ofmachine operations, the score indicating the likelihood that the userwill select a corresponding machine operation keyword; storage means forstoring said at least one keyword and scores for said at least onekeyword for each machine operation, said scores indicating thelikelihood that a user wishes to select a machine operation havingcaused a said keyword to be input; said processing means being adaptedto use the determined input keywords and said keyword storage means toadjust said scores in said score storage means, to use the adjustedscores to identify said machine operation to be executed, and to adjustthe scores for keywords stored in said keyword storage means using saidat least one keyword of said bookmark identifier.
 4. A machine accordingto claim 1, wherein said processing means is adapted to automaticallydetermine said bookmark identifier using the response to an initialquestion when a said machine operation is selected to be executed.
 5. Amachine according to claim 1, wherein said inputting means is adapted toinput a bookmark instruction and a said bookmark identifier, saidprocessing means being responsive to said bookmark instruction to storethe current set of data determined from said at least one input answeridentified by said bookmark identifier.
 6. A machine according to claim5, including keyword determining means for determining keywords usingthe input answers, wherein said set of data comprises at least onekeyword.
 7. A machine according to claim 6, including score storagemeans for storing a score of each of a plurality of machine operations,the score indicating the likelihood that the user will select acorresponding machine operation, keyword storage means for storing saidat least one keyword and scores for said at least one keyword for eachmachine operation, said scores indicating the likelihood that a userwishes to select a machine operation having caused a said keyword to beinput, said processing means being adapted to use the determined inputkeywords and said keyword storage means to adjust said scores in saidscore storage means, to use the adjusted scores to identify said machineoperation to be executed, and to adjust the scores for keywords storedin said keyword storage means using said current set of data identifiedby said bookmark identifier.
 8. A machine according to claim 1 whereinsaid processing means is adapted to execute a said machine operationwhen said set of data uniquely identifies said machine operation.
 9. Amachine according to claim 1 wherein said processing means is adapted todetermine said set of data from pieces of input information.
 10. Amachine according to claim 9 wherein said processing means is adapted toexecute a said machine operation when said set of data is complete. 11.A machine according to claim 1 wherein said processing means is adaptedto determine said set of data by looking up data associated with anoutput question using the input answer.
 12. A machine according to claim1 wherein said processing means is adapted to select a question foroutputting by said outputting means in dependence upon previous answersreceived.
 13. A machine according to claim 1 wherein said processingmeans is adapted to select a question for outputting by said outputtingmeans from a list of questions.
 14. A method of providing a machineinterface to allow a user to select a machine operation, the methodcomprising: outputting questions to the user; receiving input answers tothe questions; determining a set of data from at least one said inputanswer; using the set of data to execute a machine operation ordetermining and outputting a question to the user; wherein a set of datadetermined from at least one input answer identified as a bookmark isstored and said set of data is retrieved when the bookmark identifier isdetermined from a said input answer.
 15. A method according to claim 14including determining keywords using the input answers, wherein said setof data comprises at least one keyword and said bookmark identifiercomprises said at least one keyword.
 16. A method according to claim 15including providing a store with scores for a plurality of machineoperations, each score indicating the likelihood that the user willselect a corresponding machine operation; providing a store of said atleast one keyword and scores for said at least one keyword for eachmachine operation, said scores indicating the likelihood that a userwishes to select a machine operation having caused a said keyword input;using the determined input keywords to look up scores stored forkeywords to adjust said scores for each machine operation; using theadjusted scores to identify said machine operation to be executed; andadjusting the scores for stored keywords using said at least one keywordof said bookmark identifier.
 17. A method according to claim 14 whereinsaid bookmark identifier is automatically determined using the responseto an initial question when a said machine operation is selected to beexecuted.
 18. A method according to claim 14 including inputting abookmark instruction and a said bookmark identifier, wherein the currentset of data determined from said at least one input answer identified bysaid bookmark identifier is stored in response to the input of saidbookmark instruction.
 19. A method according to claim 18 includingdetermining keywords using the input answers, wherein said set of datacomprises at least one keyword.
 20. A method according to claim 19wherein a score for each of a plurality of machine operations is stored,the score indicating the likelihood that the user will select acorresponding machine operation; said at least one keyword and scoresfor said at least one keyword for each machine operation are stored,said scores indicating that the likelihood that the user wishes toselect a machine operation having caused a said keyword to be input; thedetermined input keywords are used to look up keyword scores to be usedto adjust said scores for said machine operations; the adjusted scoresare used to identify said machine operation to be executed; and thestored scores for keywords are adjusted using said current set of dataidentified by said bookmark identifier.
 21. A method according to claim14 wherein a said machine operation is executed when said set of datauniquely identifies said machine operation.
 22. A method according toclaim 14 wherein said set of data is determined from pieces of inputinformation.
 23. A method according to claim 22 wherein a said machineoperation is executed when said set of data is complete.
 24. A methodaccording to claim 14 wherein said set of data is determined by lookingup data associated with an output question using the input answer.
 25. Amethod according to claim 14 wherein a question is selected foroutputting in dependence upon previous answers received.
 26. A methodaccording to claim 14 wherein a question is selected for outputting froma list of questions.
 27. A program code for controlling a processor toimplement the method of claim
 14. 28. A carrier medium carrying theprogram code according to claim
 27. 29. A machine having a machineinterface to allow a user to select a machine operation, the machineinterface comprising: question storage means for storing a plurality ofquestions for output to the user; score storage means for storing ascore for each of a plurality of machine operations, said scoreindicating the likelihood that the user will select a correspondingmachine operation; question selection means for selecting a nextquestion for output to the user from said question storage means bydetermining, for each of a plurality of said questions, an average ofthe least number of questions required to be answered by the user toarrive at each said machine operation weighted by the respective scores,and selecting a question having the lowest average number; outputtingmeans for outputting the selected question to the user; inputting meansfor receiving an input answer to the question from the user; andprocessing means for responding to the input answer by carrying out asaid machine operation and/or by adjusting the scores for each of theplurality of machine operations stored in said operation storage means;said question selection means being adapted to carry out at least onefurther selection of a said next question using the adjusted scoresstored in said operation storage means for output by said outputtingmeans.
 30. A machine according to claim 29 wherein said question storagemeans is adapted to store, for a specified answer, for each of aplurality of said questions, an identifier for a corresponding machineoperation to be carried out in response to input of said specifiedanswer, and said processing means is responsive to a said specifiedanswer to a said question to carry out the machine operation identifiedby a corresponding said identifier for the specified answer.
 31. Amachine according to claim 29 wherein said processing means isresponsive to the input answer to carry out a said machine operationhaving the most significant score stored in said operation storagemeans.
 32. A machine according to claim 29 wherein said question storagemeans is adapted to store expected answers to said questions from theuser, and said question selection means is adapted to determine saidleast number of questions by predicting the expected answers input bythe user to select each of said machine operations.
 33. A machineaccording to claim 29 including word storage means for storing keywordsfor each of said plurality of machine operations; and keyworddetermining means for determining keywords using said input answer;wherein said processing means is adapted to match the determinedkeywords to the keywords stored in said word storage means, and toadjust the scores for each of the plurality of machine operations independence upon the matching.
 34. A machine according to claim 33wherein said word storage means is adapted to store scores for thekeywords for each of said plurality of machine operations, and saidprocessing means is adapted to determine scores for determined keywordsfor each of said plurality of machine operations by matching thedetermined keywords to the keywords stored in said word storage means,and to adjust the scores for each of said plurality of machineoperations using the determined scores for keywords.
 35. A machineaccording to claim 33 wherein said question storage means is adapted tostore keywords associated with expected answers to at least some of thequestions, and said keyword determining means is adapted to determinekeywords from the association with an input answer using said questionstorage means.
 36. A machine according to claim 29 wherein said questionselection means is adapted to use a recursive process for thedetermining process to identify sequences of questions to select eachsaid machine operation.
 37. A machine according to claim 36 wherein saidquestion selection means is adapted to carry out the recursive processfor each sequence until the sequence length reaches a threshold length.38. A machine according to claim 29 wherein said question selectionmeans is adapted to perform the determining for each of a plurality ofquestions, by determining an average of the least number of questionsrequired to be answered to arrive at only the machine operations havingthe highest scores weighted by the respective scores.
 39. A machineaccording to claim 29 wherein said question selection means is adaptedto perform the determining only for questions the answers to which cancause the score of a most likely machine operation to increase.
 40. Amachine according to claim 29 wherein said question selection means isadapted to select a plurality of said questions for use as the pluralityof questions in the determining process by selecting a plurality ofquestions assigned the highest score, and to determine scores for thequestions by using at least one of three techniques, namely; I. takingthe score of the machine instruction having the highest score afterasking the question and predicting an answer; II assigning a high scoreto questions relating to the same topic as a previous input answer, andIII assigning a high score to questions relating to the same topic asany previous answers.
 41. A machine according to claim 40 wherein saidquestion selection means is adapted to determine scores for thequestions by using all three techniques and taking a weighted average ofthe determined scores.
 42. A machine according to claim 29 wherein saidscore storage means is adapted to store scores for hierarchicalclassifications of said machine operations, each hierarchicalclassification comprising a topic to which the machine operations in thehierarchy below relate and having a score comprising the sum of thescores for the machine operations in the hierarchy below, the machineinterface including uncertainty means for indicating uncertainty to auser if the score for any of the hierarchical classifications atpredetermined level of hierarchical classification is below a threshold.43. A machine according to claim 29 including means for uniformlydecaying the scores for each said machine operation stored in said scorestorage means by a predetermined amount after a question has beenanswered.
 44. A machine according to claim 29 including means to allowquestions to be entered into or adjusted in said question storage means.45. A machine according to claim 29 including means to allow scores tobe entered into or adjusted in said score storage means.
 46. A machineaccording to claim 33 including means to allow words to be entered intoor adjusted in said word storage means.
 47. A machine according to claim29 wherein said outputting means is adapted to generate speech and saidinputting means is adapted to recognise speech.
 48. A method ofproviding a machine interface to allow a user to select a machineoperation, the method comprising: providing a stored plurality ofquestions for output to the user; providing a stored score for each of aplurality of machine operations, each score indicating the likelihoodthat the user will select a corresponding machine operation; selecting anext question for output to the user from the stored questions bydetermining, for each of a plurality of said questions, an average ofthe least number of questions required to be answered by the user toarrive at each said machine operation weighted by the respective scores,and selecting a question having the lowest average number; outputtingthe selected question to the user; receiving an input answer from theuser; and responding to the input answer by carrying out a said machineoperation and/or by adjusting the stored scores for each of theplurality of machine operations; and repeating the selecting step usingthe adjusted scores and subsequently repeating the outputting, receivingand responding steps.
 49. A method according to claim 48, wherein for aspecified answer, for each of a plurality of said questions, anidentifier for a corresponding machine operation to be carried out inresponse to input of said specified answer is stored, and in response tosaid specified answer being received from the user to a said question,the machine operation identified by a corresponding said identifier fora specified answer is executed.
 50. A method according to claim 48wherein a said machine operation having a stored score which is of athreshold significance is executed in response to the input answer. 51.A method according to claim 48 wherein expected answers to saidquestions from the user are stored with the questions, and in saidselecting step, the least number of questions is determined bypredicting the expected answers input by the user to select each of saidmachine operations.
 52. A method according to claim 48 includingproviding stored keywords for each of said plurality of machineoperations, and determining keywords using said input answer, whereinthe input answers are responded to by matching the determined keywordsto the stored keywords and adjusting the scores for each of theplurality of machine operations in dependence upon the matching step.53. A method according to claim 52, wherein scores for the keywords foreach of said plurality of machine operations is stored, scores fordetermined keywords for each of said plurality of machine operations aredetermined by matching the determined keywords to the stored keywords,and the scores for each of said plurality of machine operations areadjusted using the determined scores for keywords.
 54. A methodaccording to claim 52 wherein keywords associated with expected answersto at least some of the questions are stored, and the step ofdetermining keywords comprises determining keywords from the associationof stored keywords with an input answer.
 55. A method according to claim48 wherein the selecting step comprises a recursive process to identifysequences of questions to select each said machine operation.
 56. Amethod according to claim 55, wherein the selecting step carries out therecursive process for each sequence until the sequence length reaches athreshold length.
 57. A method according to claim 48 wherein in theselecting step the determination for each of a plurality of questions iscarried out by determining an average of the least number of questionsrequired to be answered to arrive at only the machine operations havingthe highest scores weighted by the respective scores.
 58. A methodaccording to claim 48 wherein the selecting step performs thedetermination only for questions the answer to which can cause the scorefor a most likely machine operation to increase.
 59. A method accordingto claim 48 wherein said selection step selects a plurality of saidquestions for use as the plurality of questions in the determination byselecting a plurality of questions assigned the highest score, anddetermines scores for the questions by using at least one of threetechniques, namely: (i) taking the score of the machine instructionhaving the highest score after asking the questions and predicting ananswer, (ii) assigning a high score to questions relating to the sametopic as a previous input answer, and (iii) assigning a high score toquestions relating to the same topic as any previous input answers. 60.A method according to claim 59, wherein said selecting step determinesscores for the questions by using all three techniques and taking aweighted average of the determined scores.
 61. A method according toclaim 48 wherein scores for hierarchical classifications of said machineoperations are stored, each hierarchical classification comprising atopic to which the machine operations in the hierarchy below relate andhaving a score comprising the sum of the scores for the machineoperations in the hierarchy below; the method including indicatinguncertainty to a user if the score for any of the hierarchicalclassifications at a predetermined level of hierarchical classificationis below a threshold.
 62. A method according to claim 48 includinguniformly decaying the scored scores for each said machine operation bya predetermined amount after a question has been answered.
 63. A methodaccording to claim 48 including receiving and storing new questions, orreceiving instructions to adjust stored questions.
 64. A methodaccording to claim 48 including receiving and storing new scores for newmachine operations, or instructions to adjust stored scores for currentmachine operations.
 65. A method according to claim 52 includingreceiving and storing new keywords or receiving instructions to adjuststored keywords.
 66. A method according to claim 48 wherein theoutputting step includes the generation of speech and the inputting stepincludes the recognition of speech.
 67. Program code for controlling aprocessor to carry out the method of claim
 48. 68. A carrier mediumcarrying the program code according to claim
 67. 69. A machine having amachine interface to allow a user to select a machine operation, themachine interface comprising: score storage means for storing a scorefor each of a plurality of machine operations, said score indicating thelikelihood that the user will select a corresponding machine operation;question selection means for selecting a next question for output to theuser from a question storage means by determining, for each of aplurality of said questions, an average of the least number of questionsrequired to be answered by the user to arrive at each said machineoperation weighted by the respective scores, and selecting a questionhaving the lowest average number; outputting means for outputting theselected question to the user; inputting means for receiving an inputanswer to the question from the user; and processing means forresponding to the input answer by carrying out a said machine operationand/or by adjusting the scores for each of the plurality of machineoperations stored in said operation storage means; said questionselection means being adapted to carry out at least one furtherselection of a said next question using the adjusted scores stored insaid operation storage means for output by said outputting means; andsaid processing means further being adapted to store a set of datadetermined from at least one said input answer identified as a bookmarkand to retrieve said set of data when the bookmark identifier isdetermined from a said input answer.
 70. A method of providing a machineinterface to allow a user to select a machine operation, the methodcomprising: providing a stored score for each of a plurality of machineoperations, each score indicating the likelihood that the user willselect a corresponding machine operation; selecting a next question foroutput to the user from a plurality of stored questions by determining,for each of a plurality of said questions, an average of the leastnumber of questions required to be answered by the user to arrive ateach said machine operation weighted by the respective scores, andselecting a question having the lowest average number; outputting theselected question to the user; receiving an input answer from the user;and responding to the input answer by carrying out a said machineoperation and/or by adjusting the stored scores for each of theplurality of machine operations; and repeating the selecting step usingthe adjusted scores and subsequently repeating the outputting, receivingand responding steps, which method further comprises identifying whetheran input answer corresponds to a stored bookmark and when the inputanswer is identified as corresponding to a stored bookmark, retrieving aset of data corresponding to that bookmark.
 71. Program code forcontrolling a processor to carry out the method of claim
 70. 72. Acarrier medium carrying the program code according to claim 71.